Conceptual Primitive Decomposition for Knowledge Sharing via Natural Language

Natural language is an ideal mode of interaction and knowledge sharing between intelligent computer systems and their human users. But a major problem that natural language interaction poses is linguistic variation, or the paraphrase problem : there are a variety of ways of referring to the same idea. This is a special problem for intelligent systems in domains such as information retrieval, where a query presented in natural language is matched against an ontology or knowledge base, particularly when its representation uses a vocabulary based in natural language. This paper proposes solutions to these problems in primitive decomposition methods that represent concepts in terms of structures reflecting low-level, embodied human cognition. We argue that this type of representation system engenders richer relations between natural language expressions and knowledge structures, enabling more effective interactive knowledge sharing

Towards Modeling Conceptual Dependency Primitives with Image Schema Logic

Conceptual Dependency (CD) primitives and Image Schemas (IS) share a common goal of grounding symbols of natural language in a representation that allows for automated semantic interpretation. Both seek to establish a connection between high-level conceptualizations in natural language and abstract cognitive building blocks. Some previous approaches have established a CD-IS correspondence. In this paper, we build on this correspondence in order to apply a logic designed for image schemas to selected CD primitives with the goal of formally taking account of the CD inventory. The logic draws from Region Connection Calculus (RCC-8), Qualitative Trajectory Calculus (QTC), Cardinal Directions and Linear Temporal Logic (LTL). One of the primary premises of CD is a minimalist approach to its inventory of primitives, that is, it seeks to express natural language contents in an abstract manner with as few primitives as possible. In a formal analysis of physical primitives of CD we found a potential reduction since some primitives can be expressed as special cases of others

Crowdsourcing Image Schemas

With their potential to map experiental structures from the sensorimotor to the abstract cognitive realm, image schemas are believed to provide an embodied grounding to our cognitive conceptual system, including natural language. Few empirical studies have evaluated humans’ intuitive understanding of image schemas or the coherence of image-schematic annotations of natural language. In this paper we present the results of a human-subjects study in which 100 participants annotate 12 simple English sentences with one or more image schemas. We find that human subjects recruited from a crowdsourcing platform can understand image schema descriptions and use them to perform annotations of texts, but also that in many cases multiple image schema annotations apply to the same simple sentence, a phenomenon we call image schema collocations. This study carries implications both for methodologies of future studies of image schemas, and for the inexpensive and efficient creation of large text corpora with image schema annotations

Linguistic Variation and Anomalies in Comparisons of Human and Machine-Generated Image Captions

Describing the content of a visual image is a fundamental ability of human vision and language systems. Over the past several years, researchers have published on major improvements on image captioning, largely due to the development of deep learning systems trained on large data sets of images and human-written captions. However, these systems have major limitations, and their development has been narrowly focused on improving scores on relatively simple “bag-of-words” metrics. Very little work has examined the overall complex patterns of the language produced by image-captioning systems and how it compares to captions written by humans. In this paper, we closely examine patterns in machine-generated captions and characterize how conventional metrics are inconsistent at penalizing them for nonhuman-like erroneous output. We also hypothesize that the complexity of a visual scene should be reflected in the linguistic variety of the captions and, in testing this hypothesis, we find that human-generated captions have a dramatically greater degree of lexical, syntactic, and semantic variation. These results have important implications for the design of performance metrics, gauging what deep learning captioning systems really understand in images, and the importance of the task of image captioning for cognitive systems researc

Script-Based Story Matching for Cyberbullying Prevention

While the Internet and social media help keep today’s youth better connected to their friends, family, and community, the same media are also the form of expression for an array of harmful social behaviors, such as cyberbullying and cyber-harassment. In this paper we present work in progress to develop intelligent interfaces to social media that use commonsense knowledge bases and automated narrative analyses of text communications between users to trigger selective interventions and prevent negative outcomes. While other approaches seek merely to classify the overall topic of the text, we try to match stories to finer-grained “scripts” that represent stereotypical events and actions. For example, many bullying stories can be matched to a “revenge” script that describes trying to harm someone who has harmed you. These tools have been implemented in an initial prototype system and tested on a database of real stories of cyberbullying collected on MTV’s “A Thin Line” Web site

Image Schemas and Conceptual Dependency Primitives: A Comparison

A major challenge in natural language understanding research in artificial intelligence (AI) has been and still is the grounding of symbols in a representation that allows for rich semantic interpretation, inference, and deduction. Across cognitive linguistics and other disciplines, a number of principled methods for meaning representation of natural language have been proposed that aim to emulate capacities of human cognition. However, little cross-fertilization among those methods has taken place. A joint effort of human-level meaning representation from AI research and from cognitive linguistics holds the potential of contributing new insights to this profound challenge. To this end, this paper presents a first comparison of image schemas to an AI meaning representation system called Conceptual Dependency (CD). Restricting our study to the domain of physical and spatial conceptual primitives, we find connections and mappings from a set of action primitives in CD to a remarkably similar set of image schemas. We also discuss important implications of this connection, from formalizing image schemas to improving meaning representation systems in AI

Grouped Variable Model Selection for Heterogeneous Medical Signals

ABSTRACT We explore statistical regression techniques for use in medical monitoring and telehealth applications. Medical embedded systems of the present and future are recording vast sets of data related to medical conditions and physiology. In this paper, distributed time-lag linear models are proposed as a means to help explain relationships between two or more medical and physiological measurements. The issues associated with performing multiple regression with heterogeneous medical data are treated as problems in model selection. An automatic method of model selection is proposed to construct models for high sample rate data by grouping sets of predictor variables. The grouped predictor variable model optimization problem is formalized. Once an initial regression is performed on all available variables, our approximate algorithm for finding the grouped variable model with the greatest validity runs in O(n 2 ) time, where n is the number of available predictor variables. This is compared to the all subsets technique which requires O(2 n ) time for the same predictor set. In our experiments with medical signal data, we find that the method produces models with reasonable goodness of fit scores and high average confidence levels for grouped predictors

Interface Design for Unmanned Vehicle Supervision through Hybrid Cognitive Task Analysis

While there is currently significant interest in developing Unmanned Aerial Systems (UASs) that can be supervised by a single operator, the majority of these systems focus on Intelligence, Surveillance, and Reconnaissance (ISR) domains. One domain that has received significantly less attention is the use of multiple UASs to insert or extract supplies or people. To this end, MAVIES (Multi-Autonomous Vehicle Insertion-Extraction System) was developed to allow a single operator the ability to supervise a primary cargo Unmanned Aerial Vehicle (UAV) along with multiple scouting UAVs. This paper will detail the development of the design requirements generated through a Hybrid Cognitive Task Analysis (hCTA) and the display that resulted from these efforts. A major innovation in the hCTA process in this effort was the alteration of the traditional decision ladder process to specifically identify decision-making tasks that must be augmented with automation

Modeling the Impact of Operator Trust on Performance in Multiple Robot Control

We developed a system dynamics model to simulate the impact of operator trust on performance in multiple robot control. Analysis of a simulated urban search and rescue experiment showed that operators decided to manually control the robots when they lost trust in the autonomous planner that was directing the robots. Operators who rarely used manual control performed the worst. However, the operators who most frequently used manual control reported higher workload and did not perform any better than operators with moderate manual control usage. Based on these findings, we implemented a model where trust and performance form a feedback loop, in which operators perceive the performance of the system, calibrate their trust, and adjust their control of the robots. A second feedback loop incorporates the impact of trust on cognitive workload and system performance. The model was able to replicate the quantitative performance of three groups of operators within 2.3%. This model could help us gain a greater understanding of how operators build and lose trust in automation and the impact of those changes in trust on performance and workload, which is crucial to the development of future systems involving human-automation collaboration

GenderMag: A Method for Evaluating Software’s Gender Inclusiveness

In recent years, research into gender differences has established that individual differences in how people problem-solve often cluster by gender. Research also shows that these differences have direct implications for software that aims to support users’ problem-solving activities, and that much of this software is more supportive of problem-solving processes favored (statistically) more by males than by females. However, there is almost no work considering how software practitioners—such as User Experience (UX) professionals or software developers—can find gender-inclusiveness issues like these in their software. To address this gap, we devised the GenderMag method for evaluating problem-solving software from a gender-inclusiveness perspective. The method includes a set of faceted personas that bring five facets of gender difference research to life, and embeds use of the personas into a concrete process through a gender-specialized Cognitive Walkthrough. Our empirical results show that a variety of practitioners who design software—without needing any background in gender research—were able to use the GenderMag method to find gender-inclusiveness issues in problem-solving software. Our results also show that the issues the practitioners found were real and fixable. This work is the first systematic method to find gender-inclusiveness issues in software, so that practitioners can design and produce problem-solving software that is more usable by everyone